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ABSTRACT 



Neural networks have demonstrated the potential to deal 
with problems related to human cognition that to date have 
stymied researchers: problems that traditional rule and 
logic based artificial intelligence techniques have been 
unable to solve. 

The range of possible neural network applications is 
not yet fully understood. To date, research and development 
has tended to concentrate on a most impressive array of 
real-time embedded systems such as speech processors, target 
recognition and robotics control systems. The Department of 
Defense is a strong proponent of this technology. 

There is another side to the development and practical 
use of neural network applications. Inexpensive software is 
presently available that allows virtually any personal 
computer to function as a simulated neurocomputer. 

This thesis - presented in a personal computer based 
hypertext environment (a diskette is available) - reports 
the results of an exhaustive literature search and extensive 
hands-on experience with these tools and techniques. 
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THESIS DISCLAIMER 

The reader is cautioned that the computer programs and 
procedures described in this research have not been 
exercised for all cases of interest. While every effort has 
been made, within the time available, to ensure that they 
are free of errors in concept, design and implementation, 
they cannot be considered validated. Any application of 
these methods is at the risk of the user. 

The diskette containing the hypertext thesis may be 
copied for distribution provided that credit is given to the 
author. Due to copyright restrictions, the BrainMaker™ 
software - a backpropagation based neural network - is not 
included on the diskette. It is required only for the 
demonstration section of the thesis. 
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I. 



INTRODUCTION 



A . BACKGROUND 

This thesis is the result of an exhaustive literature 
search coupled with extensive hands-on experience with many 
of the methods described. Despite the fact that the seminal 
theories surrounding neural networks were published over 
thirty years ago, "connectionism" is an immature, but very 
vibrant, inter-disciplinary field. It encompasses cognitive 
theory, decision theory, logic, computer science, 
mathematics, statistics, operational analysis and more. 
Neural networks have the potential to deal with problems 
related to human cognition that to date have stymied 
artificial intelligence researchers. The intent of this 
thesis is not however, to wantonly proselytize the virtues 
of neural network techniques, but rather to inform a diverse 
readership of their characteristics and to encourage further 
experimentation . 

B. SCOPE AND LIMITATIONS 

Due to heightened interest, the corpus of theoretical 
information pertaining to neural networks is exploding. New 
methods are published monthly and older ones are continually 
being updated, refined and augmented. It is impossible to 
accurately synopsize such a volatile target. The idea of 
mimicking the function of the brain is by itself 
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fascinating, but even in its embryonic phase, this 
technology is capable of addressing some vexing real world 
problems. These problems range from the control of 
supersonic aircraft to gleaning information from diverse and 
complex data. It is in this latter domain that this paper 
will focus. A variety of diverse methodologies will be 
discussed, but only one problem solving approach will be 
examined in detail. Though personal computer based neural 
network simulations are not as fast as networks implemented 
directly in silicon, they are capable of supporting the 
efficient solution of problems that do not yield to 
statistical regression or optimization. In the course of 
this thesis a representative sample problem will be 
undertaken and the reader will be afforded the opportunity 
to interactively participate in its solution. This thesis 
is intended to stimulate further research and exploitation 
of these techniques, but it cannot hope to provide an 
authoritative procedural approach to their use. 
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II. ORGANIZATION 



A. WHY HYPERTEXT? 

This typewritten paper document serves only to 
introduce and explain the motivations and methods behind the 
actual thesis, which has been written in a PC-based 
hypertext medium (the diskette is available). Hypertext was 
deemed appropriate for the following reasons: 

-it enhances the comprehensibility of the material; 

-it appeals to a broad range of readers; 

-it efficiently integrates text and graphics; 

-it provides for an actual working demonstration of the 
neural network methods described; 

-it simplifies duplication, distribution and storage 
of the document. 

In order to understand the uniqueness and versatility 
of hypertext methods it will prove useful to contrast them 
with more familiar forms of printed text. This section 
briefly highlights the benefits of using hypertext. The 
reader interested in hypertext should examine Reference 1. 

Traditionally text is, by its nature, sequential. The 
author assumes the responsibility for defining the order in 
which his work will be read. The reader proceeds 
systematically from word to word, sentence to sentence, and 
page to page until finished. Such a document is both 



3 



logically and physically linear. In contrast, hypertext is 
non-sequential . It not only permits, but also encourages 
the reader to decide the sequence that suits him. This is 
not altogether a new idea, as manual forms of hypertext have 
existed for centuries. An encyclopedia is a good example. 
Each article contains references to other related articles 
offering the reader the option to follow a "link" to 
peripheral information. In this case, navigation is part of 
the reading process. 

A computer running appropriate software can readily 
implement these hypertext techniques. Using just the 
keyboard (or other input devices), the reader may meander 
through a document at his discretion, assimilating the 
material in an order and context that suit him. To prevent 
the reader from getting disoriented, navigational tools are 
in integral part of the document. He can follow a 
prearranged tour, locate his present position in the 
document on a graphic map, and recall all of the steps that 
he has taken, so that he can ascertain where he has been and 
how much of the document he has read. 

It is not difficult to see the appeal of such a system: 
both the author (s) and the readers are afforded considerable 
flexibility. A properly designed hypertext document is 
capable of imparting at least as much information as 
conventional text and probably a lot more. 
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Granted, this technique has technological, practical 
and aesthetic advantages, but in many situations it is 
simply the most appropriate way to present the material. 
Perhaps more importantly, due to its compellingly 
interactive nature, readers tend to enjoy it. [Ref. 1] 
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B. HYPERWRITER!™ SPECIFICS 

The main portion of this thesis, contained on the 
available computer diskette, was written using a 
commercially available hypermedia authoring software 
entitled HyperWriter!™. This authoring system offers the 
developer the ability to integrate text, graphics, video and 
audio as well as hypertext "links" into a single document. 
The result of using HyperWriter!™ on a body of information 
is an interactive document that a user can explore in a non- 
linear fashion. 

For the author, this represents a departure from 
traditional techniques of composition. Disparate elements 
can be located where appropriate, and the reader can be 
provided access to this information along several very 
different paths. Composing a document using these methods 
has a decidedly different feel to it, one acknowledges the 
responsibility to not only inform the reader, but to guide 
him. 

The HyperWriter!™ software contains features necessary 
for the creation of attractive and effective hypertext 
documents. It includes a text editor with a built in 
spelling checker, and the ability to import pre-existing 
text and graphics files. Most importantly, its companion 
"reader" software, HyperReader™, is licensed for unlimited 
distribution at no cost to the reader, much simplifying the 
process of distributing the document. [Ref. 2] 
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C. DESIGN CONSIDERATIONS 

Despite the freedom of choice that hypertext affords 
the reader, to some extent the author maintains the ability 
to influence the manner in which the document is read. If 
the author deems it important for the reader to view a 
particular portion of the text in its entirety, then it 
should be presented in that way. On the other hand, the 
reader can and should be encouraged to explore when sequence 
or cohesiveness is less vital. The reader's background 
knowledge of the subject area must be taken into account so 
that terminology can be used without the need to embed 
detailed explanations. Any confusing or ambiguous terms can 
be defined through a link, thus allowing the knowledgeable 
reader to avoid the tedium of reading descriptions of 
familiar terms, while the uninformed reader has free and 
ready access to those descriptions. This in turn effects 
the length of any given portion of the document. The 
individual portions can be made much more terse while still 
conveying all necessary information. Properly executed, the 
same information can appeal to a far wider readership. 

Every effort must be made to establish meaningful 
conventions and to adhere to them consistently. In this 
thesis all links to graphic images are colored green, all 
references are red and any item with a rectangular box 
surrounding it leads to a link. In a document of this kind 
the reader will generally be presented with an overwhelming 
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amount of new information, consequently his interface to the 
material should become transparent to him and should not be 
allowed to intrude or to disconcert him. Figure 1 
graphically illustrates the simplicity of the hypertext 
interface. 



TYPICAL HYPERTEXT SCREEN 



1.0 History of Neural Networks 

Z.O Neural Network Nethodol ogles' 

3.0 Neural Netuorks and the DOD 

4.0 ft Neural Decision Support System 

5.0 Conclusions 

6.0 Bibliography' 




Figure 1 A typical hypertext screen 



In the interest of conventionality, this hypertext 
document has been constructed to appear, at least in part, 
as a traditional typewritten research paper. Essentially, 
this means that the text is presented in a fairly linear 
fashion and all of the linkages extend from the text. 
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A more powerful alternative may be to organize the 
material visually, through the use of on-screen graphical 
images or maps. The links extending from these graphics 
would lead the reader to accompanying explanatory text. 
This method proved effective in some areas of this thesis, 
but the underlying design philosophy was generally to 
emphasize links from text to graphics. 
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D . SYSTEM REQUIREMENTS 

This hyper-document is designed to be read on a 
typically configured personal computer. 

Specifically the requirements include : 

-IBM Disk Operating System (DOS) version 3.0 or later 
-a color monitor supporting Expanded Graphics Array 
(EGA) or Video Graphics Array (VGA) 

-a hard disk drive with approximately 2.5 megabytes of 
available storage space 
-a Microsoft™ compatible mouse 

-a minimum of 520 kilobytes of available system memory 
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E. INSTALLATION 

The distribution diskette is available free of charge. 

If you desire a copy please mail a formatted high density 

diskette along with sufficient return postage to: 

Prof. Tung Bui, Code AS/BD 
Naval Postgraduate School 
Monterey, CA 93943-5000 

An automated utility has been created to make the 
installation of this document as trouble free as possible. 

It is designed to copy the files from a high density (either 
5 1/4 inch 1.2 megabyte or 3 1/2 inch 1.44 megabyte) floppy 
disk drive labeled A: to the hard disk drive labeled C: . 

To install : 

-insert the floppy disk in the A: drive 
-type A: and press Enter 
-type INSTALL and press Enter 

-the program will instruct you to type START and press 
Enter to begin reading 

Note: If your system's disk drives are not labeled in this 
way you must manually install the software. This will 
require you to create a subdirectory on the chosen hard 
disk, copy all of the files from the floppy diskette to that 
subdirectory, decompress the files and finally type START. 
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III. NEURAL NETWORK DECISION SUPPORT FOR THE DOD 



A. SUMMARY (HYPERTEXT VERSION) 

Neural network methods are simply too enticing to 
ignore. Despite their current popularity among some 
academics and much of industry, their widespread use is at 
best still a few years away. The groundwork has been laid, 
but further research must contend with a significant number 
of theoretical and practical issues. Nascent neural systems 
have been denounced as being "toy'' solutions to "toy" 
problems. In part this is a valid criticism. At this 
evolutionary stage, most real-world problems remain slightly 
out of reach. This is at least partially due to the fact 
that real-world data is rarely maintained in a form readily 
suitable for neural network use. At issue too, are the 
networks' ability to grow in size and complexity, because 
most of the current architectures slow down unacceptably 
when presented with complex problems. Research in the area 
of hybrid networks is in its infancy, but it certainly 
offers tremendous promise for dealing with increasingly 
complex cognitive problems. 

Another area of concern must be the hyperbole that 
surrounds neural networks. Virtually all of the commercial 
software simulation products are guilty of making some 
pretty outlandish claims. There is a very real danger that 
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the technology will be oversold and will subsequently 
disappoint and frustrate its user base. Rule based expert 
systems have set the historical precedent. The general 
population has been promised "thinking" computers for more 
than thirty years: the mounting disappointment has created a 
generation of skeptics. Perhaps this is a blessing in 
disguise. If neural networks survive this skepticism and 
scrutiny, they will be more likely to emerge as mature and 
useful systems. 

There are immediate dangers if poorly designed and 
inadequately tested neural network applications are rushed 
into service. Such systems may glibly output dangerously 
incorrect answers. The question of legal liability has 
already been raised with regard to expert systems. Ethical 
questions will also arise regarding the invasion of personal 
privacy. Thoughtful policy guidelines and even legislation 
will eventually be required. In any event, a thoroughly 
conservative approach should be applied. Such an approach 
will demand both rigor and patience, but will provide the 
most lasting benefits. 

In order for this technology to flourish, the future 
user population must be given a rudimentary education in its 
inherent capabilities. This will serve two purposes. First, 
it will sensitize potential users to the type and format of 
data that is most usable by neural network applications. No 
problem can be solved by a neural network if the relevant 
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data is insufficient. With a little bit of foresight, all 
future databases will be ripe for neural analysis. Secondly, 
if the data exists in the proper form, the users will clamor 
for new analytical tools. 

In this era of federal fiscal austerity it may be 
necessary to divert research funding from other programs, 
perhaps even from some of the expensive real-time neural 
systems. Due to the almost unlimited spectra of 
applications, it may be possible to "piggyback" neural 
efforts onto existing projects. From a contractual point of 
view this would be desirable, because existing government 
software contracting methods don't readily apply to this 
technology. [Ref. 3] 

The Department of Defense can and should continue to 
aggressively pursue neural network research with an emphasis 
on deployable systems. The conceivable benefits of this 
leadership are enormous. 

The POD should : 

-Adopt (or create) a standard neural network 
development package suitable for the average personal 
computer user 

-Formalize the application development cycle 

-Create and man roving neural "tiger teams" with a 
charter to seek out potential application areas 

-Establish an on-line neural "clearinghouse" using the 
Defense Data Network (DDN) 

-Freely grant access to data for research purposes and 
make it available for transfer over the DDN 
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-Maintain existing expert systems for possible 
incorporation into hybrid systems 

-Completely rewrite DOD-STD 2167A, to make contracting 
for the development and purchase of these applications 
feasible 

-Examine this technology as a remedy for the ongoing 
software "crisis" because it involves no conventional 
(and expensive) programming 

- Most importantly, begin educating potential users 
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IV. CONCLUSIONS 



The promise of neural network approaches to complex 
real world problems has not gone unnoticed. Both the 
government and the civilian sector are vigorously pursuing 
its application. When compared to other areas of research, 
the level of funding required is minuscule. There are no 
guarantees of success, but the most relevant risk is not 
related to failure, it is failing to participate in future 
successes . 

As a by-product of this research, it is clear that the 
hypertext authoring environment has proven itself ideally 
suited for publications of this kind. The commercial 
software tools currently available, in conjunction with the 
general proliferation of personal computers, make this 
approach eminently practical . 
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